List 2

Question 1

a) $f(x_1, x_2) = (1-x_1)^2 + 100(x_2 - x_1^2)^2 \hspace{1cm} -10\leq x_1 \leq 10$ \, $-10\leq x_2 \leq 10$

b) $f(x_1, x_2) = x_1^2 + x_2^2 + 2x_1x_2 cos(\pi x_1 x_2) + x_1 +x_2 - 1 \hspace{1cm} |x_1| \leq 1, |x_2| \leq 1$

Question 2

Question 3 [Transfer Learning on CIFAR-10] - RESNET

Question 4 [NARX]

One step prediction $x^{(n+1)}$ from the time series $x(n) = 1 + cos(n + cos^2(n))$, where $n=0,1,2,3,...$

  1. Generate a set of samples for training, defining the prediction error to $e^{(n+1)}=x(n+1)-x^{(n+1)}$
  2. Evaluate performance by showing curve, time series, prediction curve and prediction error curve

Data

Test and train

Organizing

RNN

Question 5 [Four gaussian distributions]

Use an autoeconder network to reduce the dimensionality of data to two dimensions.

Question 6 [Next word prediction using LSTM]

Source: IG Tech Team

Download the file

Open and preprocess the data

Apply tokenization and some other changes

Creating the model

Plot the model

Train the model

Prediction

Question 7 [Transfer Learning application]

Article: Ftl-CoV19: A Transfer Learning Approach to Detect COVID-19

2.png Source: Saurabh et al. [2022]

English version

The article study a transfer learning approach to detect COVID-19. Initially, the contextualization of the face of the new coronavirus in the world since its emergence, the most common symptoms and risk factors that can cause the disease to worsen in certain individuals is presented. In this context, due to the lack of sufficient data and the high level of unpredictability, a standard model is not the best option. Thus, the study proposes the use of transfer learning, based on the pre-trained VGG-16 model, focusing on data with chest X-rays.

First, the article seeks to show the growth, challenges and consequences of COVID-19 in the world, emphasizing the role that artificial intelligence has to detect similar patterns and make predictions, including acting in the detection of diseases, in a way that can be obtained greater precision beyond the conventional tests already used. In addition, the authors present some studies carried out on a machine learning algorithm to analyze drugs and collaborate in the creation of means to contain the spread of the virus, for example, as explained by recent research carried out to detect COVID-19 through of artificial intelligence.

Thus, the approach proposed by the authors is a fine tuning transfer learning-coronavirus (Ftl-CoV19), from four phases, which involve the dataset, pre-processing, training and detection. In the first phase, the data consist of chest radiographs and CT scans, with 1,281 being COVID-19 and 1,481 being normal diagnoses. After that, there is the data pre-processing, in which the images are modeled to the desired size, from 450 x 446 to 224 x 224, and selected so that blurry or annotated images can be discarded, for example, to facilitate training and data validation. In training, CNN and the pre-trained VGG16 are used together with transfer learning and fine tuning, whose ratio used was 80 : 20 for training and testing with 2210 images for training and 552 images for validation.

In this way, the proposed approach called Ftl-CoV19 was compared with other pre-trained models, such as ResNet50, InceptionV3 and Xception under very similar test conditions, and ended up achieving better results, with training and validation accuracy of 98.82% and 99.27%, respectively.

Portuguese version

O artigo estuda uma abordagem de transfer learning para detectar COVID-19 Inicialmente, é apresentada a contextualização sobre o enfrentamento do novo coronavírus no mundo desde o seu surgimento, sintomas mais comuns e fatores de risco que podem provocar o agravamento da doença em determinados indivíduos. Dentro desse contexto, em decorrência da ausência de dados suficientes e do alto nível de imprevisibilidade, um modelo padrão acaba não sendo a melhor opção. Assim, o estudo propõe a utilização de transfer learning, a partir do modelo pré-treinado VGG-16, tendo como base raios-X de toráx.

Primeiramente, o artigo busca mostrar o crescimento, os desafios e as consequências da COVID-19 no mundo, enfatizando o papel que a inteligência artificial possui para detectar padrões semelhantes e realizar predições, incluindo atuar na detecção de doenças, de forma que se possa obter uma maior precisão para além dos testes convencionais já utilizados. Além disso, os autores apresentam alguns estudos realizados à respeito de uma algoritmo de aprendizado de máquina para analisar medicamentos e colaborar na criação de meios para conter a propagação do vírus, por exemplo, tal como explana pesquisas recentes realizadas para detectar COVID-19 por meio de inteligência artificial.

Com isso, a abordagem proposta pelos autores trata-se de uma técnica de ajuste fino de aprendizado de transferência de coronavírus 19 (Ftl-CoV19), a partir de quatro fases, as quais envolvem o conjunto de dados, o pré-processamento, o treinamento e a detecção. Na primeira fase, os dados consistem em radiografias de tórax e tomografias computadorizadas, com a presença de 1.281 sendo COVID-19 e 1.481 diagnósticos normais. Em seguida, tem-se o pré-processamento de dados, em que as imagens são modeladas pra o tamanho desejado, indo de 450 x 446 para 224 x 224, e selecionadas de forma que se possa descartar imagens borradas ou com anotações, por exemplo, para facilitar o treinamento e a validação dos dados. Já no treinamento, usa-se CNN e o VGG16 pré-treinado em conjunto com transfer learning e fine tuning, cuja proporção utilizada foi 80 : 20 para treinamento e teste com 2210 imagens para treinar e 552 imagens para validação.

Dessa forma, a abordagem proposta chamada de Ftl-CoV19 foi comparada com outros modelos pré-treinados, como ResNet50, InceptionV3 e Xception sob condições bastante parecidas de teste, e acabou alcançando melhores resultados, com precisão de treinamento e validação de 98,82% e 99,27%, respectivamente.

Project [Pneumonia Detection using CNN]

Source: Kaggle: Pneumonia Detection using CNN

The dataset is organized into 3 folders (train, test, val) and contains subfolders for each image category (Pneumonia/Normal). There are 5,863 X-Ray images (JPEG) and 2 categories (Pneumonia/Normal). Chest X-ray images (anterior-posterior) were selected from retrospective cohorts of pediatric patients of one to five years old from Guangzhou Women and Children’s Medical Center, Guangzhou. All chest X-ray imaging was performed as part of patients’ routine clinical care. For the analysis of chest x-ray images, all chest radiographs were initially screened for quality control by removing all low quality or unreadable scans. The diagnoses for the images were then graded by two expert physicians before being cleared for training the AI system. In order to account for any grading errors, the evaluation set was also checked by a third expert.

Loading the Dataset

Data Visualization & Preprocessing

The data seems imbalanced. Solution: Data Augmentation

Data Augmentation

For the data augmentation:

  1. Randomly rotate some training images by 30 degrees
  2. Randomly Zoom by 20% some training images
  3. Randomly shift images horizontally by 10% of the width
  4. Randomly shift images vertically by 10% of the height
  5. Randomly flip images horizontally. Once our model is ready, we fit the training dataset.

Training the Model

Analysis after Model Training

Some of the Correctly Predicted Classes

Some of the Incorrectly Predicted Classes

Project [GAN using CIFAR-10]

Source: Kaggle - Generative Adversarial Networks - Demystified

Data: CIFAR-10

Single class selected: birds

The Generator

The Discriminator

Original images

Some images produced by the generator